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1.  INTRODUCTION 


H 


Bayes  least  squares  linear  (BLSL)  estimators  were  introduced 
by  Whittle  (1957,  1958)^ ind  described  explicitly  and  further  devel¬ 
oped  by  Hartigan[(1969) .  The  method  was  applied  to  estimation  of 

coefficients  of  orthogonal  expansions  of  regression  functions  in 

*  •''"X  fKi’j  ;4  |i 

(-Brunk  ,-t980) .  xIn  -efee  present  paper  wo  observe  that  when  many  f 

^ - - be  f 

observations  are  available  we  can  expect  the  BLSL  method ^to  yield 

substantially  the  same  results  as  a  full  Bayesian  treatment;  and 

-  is  i  llv»S A- 

r  we  -irliu aerate  the  method^in  the  context  of  estimation  of  spectral 
densities.  In  that  context,  the  estimators  suggested  will  appear 
rather  ordinary.  But  they  are  not  completely  ad  hoc:  each  comes 
with  an  interpretation.  And,  when  large  samples  are  available, 
the  posterior  distribution  of  the  estimator  at  a  fixed  frequency 
is  (approximately)  normal,  with  easily  calculated  standard  deviation. 


2.  ORTHOGONAL  EXPANSIONS  AND  THE  BLSL  METHOD 

In  order  to  be  more  explicit,  we  recall  the  description  of  the 

estimators  given  in  (Brunk,  1980) .  Since  the  size  of  the  data  set 

is  relevant  here,  we  allow  the  number  of  observations  to  appear  in 

the  notation.  Thus  for  each  integer  n  we  have  a  set 

(x  ,  x, ,  ...  ,  x  }  of  values  of  an  "explanatory  variable"  in  a 
no  m  nn 

space  X  of  possible  values.  The  regression  function  R  is 

Q 


m 


iPV 


defined  on  X  and  is  assumed  to  have  a  finite  expansion  in  terms  of 

specified  functions  r  ,  R  and  {A  ,  r  *  0,  1,  . . .  ,  n}: 

no  no  nr 


l  3nr  ^nr(x)  *  x  6  X* 


The  observations  or  responses  (Y  ,  Y  _,...,  Y  )  are  assumed 

no  n  a  nn 

independent,  and  Yqj  is  assumed  to  have  mean 


E(V  ■  w 


and  variance 


Var(Y  ,)  -  1/  X  *  ,  j  -  0,  1 . n  , 

nj  n  “J 


where  ir  ,  .  v  are  known.  We  shall  assume  also  that  X 

no  nn  n 

is  known,  though  in  practice  Xq  may  be  estimated  from  the  data. 

(One  can  state  less  restrictive  assumptions  that  suffice.  In 
what  follows,  as  elsewhere,  a  tilde  underline  indicates  a  random 
entity,  and  "  is  used  between  two  expressions  when  the  left 

is  defined  by  the  right.  These  assumptions  are  that  the  linear 
expectation  of  Y^  be  RQ(*nj) s 

“^nj  l?n  "  8n^  "  Rn^xnj^  *  J"0,l,...,n, 

where  0  :■  (0  ,  ...  ,  0  )'  ;  and  the  linear  covariance  matrix 
— n  ,no  .nn 

of  Y  :»  (Y  ,  . . .  ,  Y _ ) ?  given  0  •  0  have  entries 

n  no  nn  .n  n 

"  r_(x_,)][Y_4  -  R_(x_4)]  |0_  -  0n)  -  /X.ir.4  , 


W 


here  is  the  Kronecker  delta:  6^ ,  -  1  if  i  -  j  ,  4  *  0 

if  i  +  j  .  ) 

The  functions  {$  ,  r«0,  1,  ...,n)  are  assumed  selected 

so  as  to  be  orthonormal  with  respect  to  the  design  of  the  experiment 

n 

V  it  .r  2  (x  .)  4  (x  .)  4  (x  .)  ■  6 
j«0  n0  nr  n8  ra 

The  function  Rqq  is  thought  of  as  a  prior  mean,  so  that  one  sets 

E(SnI)  -  0  ,  r  -  0,  1 . n  . 

It  is  argued  in  (Brunk,  1980)  that  since  each  coefficient  g  has 
an  interpretation  Independent  of  8  for  s  f  r,  it  may  often  be 

Q8 

reasonable  to  assign  0  :■  (0  ,  6  . . 8  )'  a  joint  prior 

,n  «ns  jtui  ,nn 

distribution  according  to  which  the  components  8QO . 8^  are 

independent;  and  we  set 


V  !-  1/V«(6dc>  ,  t-0.  . 


Set 


Unr  irnjrnoCxnj)  *nr<Xnj)[Ynj  "  Rno(xnj)] 


^.0  CnrjWnj  ’  *  "  °*  1 . n  • 


c  -4  i  r  (x  .)  +  (x  .) 

nrj  nj  no  nj  nr  nj 


(2.8) 


where 


That  is,  the  random  vector 


U 

n 


U 

nr 


U 

nn 


)’ 


is  obtained  by  applying  the  orthogonal  transformation  C 

vector  W  :■  (W  ,  ...  ,V  )  *  ,  where 
n  no  nn 

^Cn*rj  cnrJ  ’  r"°»1 . .  J*0,  1,  . 


It  follows  from  (2.4)  that 


E(U  1 8  -  S  )  -  S 
nr'.n  n  nr 


and  that 


tcOT(0»r-,,M)l!n 

Then  the  linear  expectation  of 
by 


*  5r»  1  Xn  • 

n  rs  n 


!n  8iven  °no  ’  ‘ 


U 

nn 


?nr  “  Xn  Unr  /  (  Xn  +  Tnr)  *  r  “  °»  1 . n 


and  the  linear  variances  and  covariances  are  given  by 


!(5nr  -  !nr>2  ■  1/U»  +  Tnt)  •  '"O'1 . n 

E(!nr  *  !nrH!a.  -  -  0  £or  r  +  *' 


For  fixed  x  ,  the  linear  expectation  of  R  (x)  is 


(2.15) 


fcV 


B 


R  (x)  -  R  (x)  +  r  (x)  T  0  &  (x) 

n  no  no'  i*  .  nr  vnr' 


no 


r-0 


;nr  Tnr 


and  its  linear  variance  Is 


E([R  (x)  -  R  (x)  ] 2) 

u  n 


no 


(X)  I 


r-0 


♦niW'<VTnr>  * 


(2.16) 


Note  that  the  method  does  not.  In  general,  provide  posterior 
covariances . 


It  will  be  useful  to  note  that  when  R  -  0  and  t  .  -  0 

no  nj 


for  j  -  0,  1,  ...  ,  n  ,  the  formula  for  R  (x)  provides  the  ordinary 

n 


least  squares  regression  of  Y  on  x  with  weights  , 


J  -  0,  1,  ...  ,  n.  Indeed,  when  k  is  fixed,  k  (-  n  ,  the 


orthogonality  properties  of  the  functions  (6nr  :  r-0, 1,  ...  ,  n} 


rk  * 

lead  to  I  a  *  as  ordinary  least  squares  estimator  of  R  , 
r-0  nr  n 


where 


nr 


J“  Vj  *nr(xnJ)Ynj  ’  r"  °»  1 . k  * 


Note  that  a  .  »  0  .  if  R  -0  and  a  -  8  if  also 
nr  nr  no  nr  nr 


Tnr  "0»  r  "  0  ,  1  »  . .  •  »  k  . 


Of  course,  the  BLSL  estimators  may  be  considered  from  a  con¬ 
ventional  point  of  view.  That  is,  one  may  choose,  as  is  customary 
when  estimating  a  regression  function  Rq  ,  some  family  of  functions 
considered  adequate  for  representing  It.  One  may  then  orthogonallze 
these  functions  with  respect  to  the  design,  to  obtain  functions 
,  r-0,  1  ,  ...  ,  n)  ;  and  then  specify  a  function  Rfto  and 


"precisions"  {t^,  r-0,  1,2,  ...  ,  n  )  ,  thus  finally  obtaining 


an  estimator  of  the  regression  function  Rq  .  But  such  an  estimator 

Is  not  completely  ad  hoc;  It  comes  with  an  interpretation.  One 

realizes  that  one  is  considering  the  same  estimator  that  another 

investigator  would  use  who  was  applying  the  Bayes  least  squares 

linear  method,  specifying  Rqq  as  a  prior  mean,  and  the 

{t  _  >  r  •  0,  1,  ...  ,  n}  as  precisions  of  the  coefficients  in  the 
nr 

expansion  of  Rq  .  And  one  may  like  to  bear  that  in  mind  when 

specifying  Rqq  and  the  precisions. 

In  principle,  an  investigator  who  is  well  acquainted  with  the 

functions  {A  }  to  be  used,  and  who  also  has  a  clear  and  definite 
nr 

opinion  as  to  the  probable  shape  of  the  regression  function  to  be 

estimated,  can  specify,  more  or  less  uniquely,  a  prior  mean  and  prior 

precisions.  But  in  practice,  there  may  be  a  rather  wide  variety  of 

specifications  that  all  seem  reasonable.  One  may  then  examine  a 

number  of  estimates  arising  from  a  range  of  "reasonable"  priors. 

As  to  the  specification  of  the  prior  mean,  R  ,  a  heuristic  argu- 

no 

ment  is  given  in  (Brunk,  1981)  that  it  is  often  reasonable  to  fit 
the  data — roughly — by  a  member  of  a  family  of  smooth  functions 
depending  on  only  one  or  two  parameters.  That  leaves  still  the 
precisions ,  tqj.  ,  r  -  0 ,  1,  ...,n.  Two  suggestions  come  from  think¬ 
ing  of  them  as  reciprocals  of  prior  variances  of  the  parameters 
§nr  *  r  -  0,  1,  2,  . . .  ,  n  . 

(i)  If  the  functions  $Qr  oscillate  more  and  more  rapidly 
with  increasing  r  ,  one  can  express  an  opinion  that  the  estimate 

A 

Rq  is  "smooth”  by  specifying  large  values  of  TQr  when  r  is  large. 
Oil)  The  precisions  should  be  specified  independently  of  the 
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3.  APPROXIMATE  NORMALITY 

Now  let  us  briefly  imagine  that  the  random  variables 

U  ,  ,Q  were  observed  (rather  than  Y  ,  . . .  ,  Y  )  ,  that  they 
no  on  no  nn 

were  independent  according  to  their  joint  distribution  given  g^  , ' 
and  that 


u  |e  -  e  ~  N(g  ,  i/x  ) . 

nr  _n  n  nr  n 


(3.1) 


Suppose  also  that  0q  is  given  a  joint  prior  distribution  according 

to  which  its  components  0  . 0  are  independent,  and 

.no  .nn 


?nr  ~  N(0  »  1/xnr}  »  r  "  °*  L*  * n  9 


(3.2) 


Then  these  components  are  also  independent  and  normally  distributed 
according  to  their  posterior  distribution,  with 


-  “no . ”n»  ‘  “on*  *  Vor'<Xn  +  rnr>  • 


(cf.  (2.12)  )  and 


(3.3) 


_  “ u  ,  ...  ,  U 
.nr  no  no 


nn 


u)  -  1/(X 
nn  n 


Tnr) 


(3.4) 


r  *  0,  1,  ...,  n 

(cf  (2.13),  (2.14)  )  . 

We  are  Interested  particularly  in  contexts  in  which  one  ex¬ 
presses  an  opinion  as  to  the  "smoothness"  of  RQ  by  assigning 
large  precisions  to  the  coefficients  of  rapidly  varying 

functions  $  in  the  expansion  of  R_  .  Then,  typically,  there 
nr  n 

is  a  positive  integer  m  such  that  the  posterior  mean  and  variance 


of  0  are  so  near  zero  for  r  >  m  that  corresponding  terms  in 
.nr 

the  expansion  can  be  neglected;  and  this  is  so  also  for  the  posterior 

linear  expectation  and  for  E(B  -  B  )2  in  the  BLSL  method. 

.nr  -nr 

When  the  observations  Y  ,  ...  ,  Y  are  not  jointly  normal, 

no  nn 

but  n  is  large,  we  shall  argue  that  often  U  ,  ...  ,U  will 

no  nn 

have  approximately  a  joint  normal  distribution  (cf.  Theorem  4.1, 

to  follow) .  The  data  Uqq  ,  . . .  ,  Unn  are  fully  equivalent  to  the 

original  data  Y  ,  . . .  ,  Y  :  each  may  be  obtained  from  the  other 

by  an  orthogonal  linear  transformation.  And  for  some  positive 

integer  m,  U  ,  ...  ,  U  may  safely  be  ignored,  so  that  if 
n,m+i  nn 

B  ,  . . .  ,  B  are  given  the  multinormal  prior  distribution  des- 
-no  -nn 

cribed  above,  then  according  to  their  joint  posterior  distribution 
they  are  (approximately)  jointly  normally  distributed  with  posterior 
means  given  by  (2.12)  and  (3.3)  and  posterior  variances  by  (2.13)  and 
(3.4).  A  theorem  (Theorem  4.1)  that  suggests  this  approximation  is 
given  in  Section  4,  and  its  proof  in  the  appendix. 

4.  ASYMPTOTIC  NORMALITY 

We  consider  triangular  arrays  V  :  V  ,  V  .,...,  V  ,  , 

no  ni  n,tcn 

n  ■  1,  2,  ...  ,  where  k  »  as  n  -»■  «  ,  and  where  E(V  .)  *  0  , 

n  nj 

Var (V^ )  «  1  ,  j  ■  0,1,  ...  ,  kQ  .  We  shall  argue  that  this  array 

is  asymptotically  normal,  given  B  ■  B  »  when 

.n  n 

V  [U  -  E(U  ) ] / [Var (U  ) ,  r  -  0,1 . k  (4. 

nr  nr  nr  nr  n 

provided  that  kQ  does  not  grow  too  fast.  We  use  Mallows's  (1972) 
definition  of  asymptotic  normality:  the  array  V  is  jointly 
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asymptotically  normal  (j.a.n.)  if  for  every  array^  u/>  of  reals 

-Ha  2 

•  $  a  . . A  ,  n  -  0,1,  ....  such  that  2,  *2  m  1  »  the 

no  nj  n^kn  ^g  nr 

r  n 

random  variable  2.  a  V  converges  in  distribution  to  the 
r-o  nr  nr 

standard  normal  distribution.  (Mallows  (1972)  observes  that  this 
implies  that  for  each  d  ,  (Vqo  ,  . . .  ,  V  ^)  converges  in  distri¬ 
bution  to  the  standard  d-dimensional  normal  distribution.) 

Theorem  4.1.  Let  there  be  a  positive  number  M  such  that 


where 


(in  V 


M 


0 


"nj  E(V  *  Rn  <V  *  j  “  °»1* 


H|  II  *  1|2|  •  •  •  •  0 


And  suppose  that 

t{/r 


k 

I  I  I  I  !  J  ■  °»1 . n  } 

nj  no  nj  ^g  nr  nj 


as  n  •  . 

Then  the  array  'V  is  j.a.n. 

The  proof,  given  in  the  appendix,  consists  of  showing  in  a 

k 

r  n 

straightforward  way  that  the  characteristic  function  of  £  a  V  , 

r-0  nr  nr 

evaluated  at  a  real  number  t  ,  converges  to  exp(-t2/2)  as 

n  ■*  • .  This  theorem  presents  an  instance  of  a  phenomenon  studied 

by  Mallows  (1969) .  Note  that  the  random  variables  U  _  are  obtained 

nr 

via  an  orthogonal  transformation  from  the  random  variables  ;  and 
that  while  these  are  independent,  they  need  not  be  normally  distri¬ 
buted.  Although  the  inverse  orthogonal  transformation  would  recover 
the  original  non-normal  random  variables,  the  random  variables  UQr 
are  *\everthel'  is  j.a.n.  Mallows  (1969)  proves  a  theorem  with  a 


A 


somewhat  stronger  conclusion,  described  in  terms  of  the  integrated 
squared  difference  between  the  standard  normal  distribution  function 
and  the  distribution  function  it  approximates.  As  stated.  Mallows' & 
theorem  requires  independent,  identically  distributed  random  vari¬ 
ables.  While  the  method  of  proof  appears  to  allow  a  relaxation  of 
that  requirement,  it  does  seem  to  require  that  the  distributions  not 
be  too  nearly  of  lattice  type.) 


5.  BLSL  ESTIMATION  OF  SPECTRAL  DENSITIES 
We  consider  the  problem  of  estimating  the  spectral  density  of 
a  stationary,  purely  nondeterministic  time  series 


X 


t 


e 

~t-s 


(not  necessarily  Gaussian) ,  where  the  random  variables  e  , 

t  *  ...  ,  -1,0,1,  ...  are  independent  with  zero  means  and  common 

variance.  The  coefficients  o  ,  s  ■  0,1,  ...  ,  are  unknown,  but  we 

s 

assume  that  |u  |  <  •  and  that 

t-0  c 

suptE[  e£2  i{|g  |>cJ]  — 4  0  as  c  — }  » 

(cf.  Anderson  (1971),  page  482).  The  spectral  density,  to  be 
estimated,  is 

f  (  w)  ■  £  °*  r(t)exp(2iriu>t)  ,  -  1/2  <  ■  u  <  ■  1/2  , 

t»  -« 

where 

r(t)  E(X  X  ) 


is  the  covariance  function.  We  follow  Wahba  (1980)  in  formulating 
the  problem  as  an  ordinary  regression  problem,  with  values  of  the 
log  periodogram  (cepstrum)  as  data.  Let  X^,  X^,  . ..,  X^Q  be 
observed .  We  have 

2n 

I  (a>)  (l/2n)  |  Y  X.  exp(2iriut)  ! 2  ,  -1/2  <«  w<«  1/2  , 

n  i-1  c 

IQj  In(j/2n)  (-In(-j/2n>) 

-  f  0/20)1^  , 

where 

Tnj  1nj/f(J/2n)  *  j  “  °’  lf  2 . “* 

Asymptotically,  as  n  -*■  «  ,  the  random  variables  ^Tnj  *  j*0,l,...,n) 
are  independent,  with  Tqq  and  distributed  as  x2(l)  *uod 

2Tqj  ~  X2C2)  for  J  *  l,2,...,n-l  (Anderson,  1971,  pp.  484-485). 

We  set 

l°*  Inj  +  CJ  *  J  ‘  0>1 . . 

where  C  -  C  :■  (In  2  +  y)  ,  C.  :■  Y,  j  -  1,2,  ...  ,  n-1  ,  and 
on  j 

where  y  is  the  Euler-Mascheroni  constant,  approximately  0.57721. 

Then  the  random  variables  {Y  j  ■  0,1,  ...  ,  n}  are  asymptotically 
independent,  with  {Yq ^ ,  j  ■  1,2,  ...  ,  n-1)  asumptotically  identically 
distributed. 

We  set 

R  (x)  log  f  (x/2)  for  0  <  -  x  <  -  1  . 
n 

According  to  the  asymptotic  distribution. 


According  to  the  asymptotic  distribution. 


E(V  ■  VJ/n)  ■  R,<V  * 

where  x  .  j/n  for  j  ■  0,  1,  2,  . . . ,  n  ,  while 

Var(YQj)  -  ir2/6  ,  J  -  1,  2 . n-1  , 

Var(Y  ,)  -  "2/2  for  j  -  0,  n  . 

We  shall  conduct  the  analysis  and  carry  out  the  computations  as  if 
Var(Y^)  ■  it  /3  for  j  ■  0,  n  ,  since  in  any  case  the  influence  of 
these  two  terms  is  negligible  for  large  n  .  In  the  notation  of 
Section  2,  then,  we  take 


*nj  !"  \m  s"1/2*  ffnj  !"  1  for  J"1*2 . n“lR 

and 

^  Zm  &  /  It  )  Q  a  1)2|  •••  e 

n 

The  functions  $  are  given  by 

♦__(*)  1/  t/n  , 

no 

k 

*  (x)  :■  (2/n)^  cos  irrx  ,  r  ■  1,2,  . . .  ,  n  ,  x  e  [0,1]  . 

nr 


(5.1) 


The  function  r^o  used  is  identically  1. 

It  is  most  convenient  to  take  as  R  a  linear  combination  of 

o 

the  functions  {$  _  i  r  ■  0,  1,  . . .  ,  n)  .  As  we  noted  earlier,  a 
nr 

heuristic  argument  is  given  in  (Brunk,  1981,  page  117)  that  it  is 
reasonable  for  the  investigator  to  select  as  prior  mean  Rq  a 
regression  function  that  is  consistent  with  both  the  data  and  his 
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opinion  as  Co  ics  shape.  In  particular,  Che  investigator  may  simply 

use  ordinary  least  squares  with  weights  tt  ,  . . .  ,  tr  to  choose 

no  nn 

coefficients  a  ,  a.  ,  . . .  ,  a.  for  k  *  1  or  2  or  3  in  fitting  the 
k  0  1  k 

function  £  a  <J>  to  the  data.  This  would  yield 
r-0  r  nr 
k 

R  (x)  -7  a  $  (x) 

0  r-0  r  nr 

where 

n 

a  :■  7  it  (x  .)Y  .  . 

r  Q  nj  nr  nj'  nj 

And  then  (cf.  the  end  of  Section  2)  if  tq  “  Ti  "  *  *  *  “  \  “  0  » 
we  have 


R  00  -  I  S'  <>  (x)  »  where 

n  xxv  nr 


X  U '  /  (X  +  T  )  ,  r  —  .0,1,  ...  ,  n  , 
n  nr  n  nr 


U'  l  w  ♦  (x  )Y  .  ,  r  -  0,1 . n. 

nr  nj  nr  nj  nj 

In  other  terms,  if  the  Investigator  chooses  to  consider  a  specifica¬ 
tion  of  prior  mean  and  precisions  that  has  Rq  as  the  ordinary 
least  squares  estimator  of  Rq  as  a  linear  function  of 
*no’  *nl . *  “d  1188  Prlor  Precisions  -  x^,  -  . . .  -  0  , 


no  nl 


then  &  is  precisely  what  it  would  be  if  he  took  R  9  0  (in  which 
n  o 

case  U  would  become  U'  )  . 
nr  nr 

The  functions  $Qr  described  above  depend— in  a  simple  way— on 

n  .  It  is  more  convenient  when  considering  specification  of  preci- 

*  h 

slons  to  use  functions  independent  of  n  :  $  :«  (n/2)  $  ,  so 

n  nr 


.V 
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*  .  if 

$o  :■  /1/2  ,  ^ (x)  :*  cos  ffrx  ,  r  >  0  . 


(5.2) 


Wc  set 


:«  (2/nKS  ,  r  -  0,  1,  ...  ,n. 
nr 


Then — n  being  fixed — we  are  assuming  that  has  an  expansion 


R  (x)  -  R  (x)  +  l  6*  $*  (x)  ,  x  e  [0,1]  . 


nr  Tt 


While  formally  the  assumed  expansion  of  R  depends  on  n  , 

n 

we  consider  that  terms  of  large  index  are  negligible;  and  each 
term  is  in  fact  independent  of  n  ,  so  that  we  write 


R(x)  -  R  (x)  +  l  6  ♦  (x)  ,  x  e  [0,1] 

°  r-0  r  r 


The  coefficients  {  0r  ,  r  »  0,1,  ...  , n  >  ,  modeled  as  random  vari¬ 
ables,  have  means 


(5.3) 


E(gr)  -  0 


and  precisions 


t*  1/Var(0*  ) 


£ 


In  the  present  context  and  notation.  Equation  (2.12)  becomes 
8nt  ■  Ur  / (n/2  +  irzTr  /  6)  , 


(5.4) 


IlVI 


where 


Ur  !"  l  (J/n)lYnj_VJ/n>>  * 

i  *° 

r  *  0,1,  . . .  ,  n 


(5.5) 


We  note  that  E(2Ur  /n|8  )  ■  8r  ,  and  that  2Ur/n  is  the  ordinary 

* 

least  squares  estimate  of  8f  with  weights  ir  ,  i-0,1,  ...  ,n. 
For  fixed  x  e  [0,1]  ,  the  posterior  linear  expectation  of  R(x)  is 

*  ^  A  A 

R  (x)  -  R  (x)  +  I  8  <p  (x) 
n'  o'  .nr  Tr 

and  its  linear  variance  is 

E[R(x)  -  R  (x)]2  -  l  [d>*(x)  ]2  /  (3n/w2  +  x*)  . 

r-0 

In  view  of  Theorem  4.1,  when  n  is  large  we  expect  the  posterior 

distribution  of  R(x)  to  be  approximately  normal  with  mean  R  (x) 

n 

given  by  (5.6)  and  variance  given  by  (5.7). 


6.  EXAMPLES 


As  a  first  example,  we  have  used  the  example  used  by  Wahba 
(1980) : 


Xt  "  E._,  \  Xt-k  +  e*  * 


k-1 


where  *  1.4256  ,  y^  “  ”0.7344  ,  y ^  -  0.1296  ,  and  where  the 
random  variables  efc  ,  t  -  ...  ,  -2,  -1,  0,  1,  2,  . . .  are  independent, 
each  having  the  standard  normal  distribution.  The  simulation  was 


carried  out  starting  with  X_^ q  ■  0  and  then  discarding 


X_29»  •••  ,  Xq  .  (These  observed  X^  are  not  to  be  confused 


with  x  .  :■  j/n  in  the  formulas  in  Section  5.)  One  set  of  256 
nj 


points  was  obtained  in  this  way  (n  ■  128) ,  as  also  a  larger  set  of 
1024  (n-512)  containing  the  first. 


:: 


16 


The  function  R  used  is  defined  by: 

o  ' 


R  (x)  ;*  0.1  +  2.9  cos(ux)  +  0,5  cos(2ttx),  0  <  x  <  1  . 
o  —  — 


The  accompanying  Tables  1  and  2,  and  Figures  1  through  14, 


relate  to  the  following  four  specifications  of  precisions 


r  m  0,  1,  ...  ,  n  : 


A:  t 


(0.21  r) 


B:  t 


(0.00024) (6.4)  , 


C:  t 


0.004(4)  , 


D:  t 


0.1(3) 


Table  1  indicates  the  "damping”  effect  of  each  specification  of 


precisions;  that  is,  the  entry  in  the  table  is  1/ (1  +  Tr2Tr  /  3n)  , 


the  factor  by  which  the  ordinary  least  squares  estimate  ,  2Ur  /  n  , 


is  multiplied  to  obtain  8r  ,  when  n  «*  128  (256  observations) . 


The  entries  in  Table  2  are  for  n  •  512  (1024  observations) . 


Each  specification  of  {  Tr  ,  r  ■  0,  1,  ...  ,  }  leads  to  a  "window 


estimator"  that  could  be  considered  from  a  conventional  point  of 


view.  Any  or  all  of  these  specifications  might  appear  reasonable 


to  an  investigator.  The  precisions  specified  under  D  increase 


most  rapidly,  and  might  be  expected  to  lead  to  the  smoothest  esti¬ 


mates  of  R  .  Initially,  the  precisions  A  increase  somewhat  more 


rapidly  than  those  of  B  ,  though  eventually  those  of  B  increase 


much  more  rapidly.  In  fact,  those  of  B  were  deliberately  selected 


(by  regression  of  log  on  r  ,  from  A)  so  as  to  be  near  those 


of  A  for  r  < -  10  . 


VO  00  NJ  O'  Ul  W  h)  H  O 
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Table  1 


Multipliers  for  n  -  128 


r 

A 

B 

c 

D 

0 

1.000 

1.000 

1.000 

0.997 

1 

1.000 

1.000 

1.000 

0.992 

2 

1.000 

1.000 

0.998 

0.977 

3 

0.999 

1.000 

0.993 

0.935 

4 

0.994 

0.999 

0.974 

0.828 

5 

0.963 

0.993 

0.905 

0.616 

6 

0.860 

0.959 

0.704 

0.348 

7 

0.641 

0.787 

0.373 

0.151 

8 

0.380 

0.365 

0.129 

0.056 

9 

0.193 

0.083 

0.036 

0.019 

10 

0.093 

0.014 

0.009 

0.007 

11 

0.046 

0.002 

0.002 

0.002 

12 

0.023 

0.001 

0.001 

13 

0.012 

0.000 

S|HSWr  T  7 

14 

0.007 

0.000 

0.000 

0.000 

15 

0.004 

0.000 

16 

0.002 

0.000 

0.000 

0.000 

17 

0.001 

0.000 

0.000 

■tWr  •  2 

18 

0.001 

0.000 

0.000 

19 

0.001 

0.000 

0.000 

Table  2 


Multipliers  for  n  -  512 


0 

1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 


A 

1.000 

1.000 

1.000 

1.000 

0.998 

0.991 

0.961 

0.877 

0.710 

0.489 

0.292 

0.161 

0.087 

0.048 

0.027 


B 

1.000 

1.000 

1.000 

1.000 

1.000 

0.998 

0.990 

0.936 

0.697 

0.265 

0.053 

0.009 

0.001 

0.000 


c 

1.000 

1.000 

1.000 

0.998 

0.993 

0.974 

0.905 

0.704 

0.373 

0.129 

0.036 

0.009 

0.002 

0.001 


D 

0.999 

0.998 

0.994 

0.983 

0.951 

0.865 

0.681 

0.416 

0.192 

0.073 

0.026 

0.009 

0.003 

0.001 


15 

16 

17 

18 

19 

20 


0.016 

0.009 

0.006 

0.004 

0.002 

0.002 


£ 

i 

*•> 

Jl 
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Figure  1  shows  both  the  true  R(x)  :■  log  f(x/2)  and  the 

A 

estimate  R  obtained  from  the  256  observations  (n  -  128)  using 
precisions  A.  Figures  2,  3,  and  4  show  estimates  obtained  through 
precisions  B,  C,  and  D  respectively.  Figure  5  shows  first  and 
fourth  estimates,  together  with  the  true  R  .  Figure  6  provides  the 
graphs  for  the  first  prior,  but  based  on  the  1024  observations 
(n  -  512) ,  and  Figure  7  is  for  the  fourth  prior .  Figure  8  shows 
first  and  fourth  together. 

Figure  9  shows  the  spectral  density  estimate  and  the  true 
spectral  density,  graphed  against  twice  the  frequency,  for  the 
first  prior  (A),  and  256  observations.  The  curves  lying  above  and 
below  the  graph  of  the  estimate  indicate  the  precision  of  the 

estimate  in  the  following  way.  For  fixed  x  ,  the  asymptotic 

* 

theory  leads  us  to  expect  R  (x)  to  be  approximately  normally 
distributed  according  to  its  posterior  distribution.  Its  (approxi¬ 
mate)  posterior  variance  is  given  by  (5.7).  Letting  o(x)  denote 
the  square  root  of  this  posterior  variance,  the  upper  and  lower 

A 

graphs  are  graphs  of  exp[R(x)  +  a  (x)j  .  Figure  10  gives  the 
same  information  for  the  fourth  prior  (D) ,  and  Figure  11  shows 
first  and  fourth  estimates  together.  Figures  12  and  13  compare 
with  Figures  9  and  10,  but  for  the  case  of  1024  observations. 
Spectral  density  estimates  for  first  and  fourth  priors  are  shown 
together  in  Figure  14,  for  the  case  of  1024  observations. 

For  a  second  example  we  have  used  underwater  ambient  noise 
data  kindly  furnished  by  the  Naval  Undersea  Warfare  Experiment 
Station  at  Keyport,  Washington.  A  sample  of  1024  observations 


(n  «  512)  was  Caken,  with  effective  sampling  frequency  80  khz.  The 

procedure  described  In  Section  5  was  followed,  and  precisions 
*  * 

Tr  :*  0.1(3  ),  r  ■  0,  1,  ...  ,  512  were  used.  Figure  15  shows  the 
estimated  log  spectral  density,  together  with  graphs  obtained  by 
adding  and  by  subtracting  the  square  root  of  the  expected  squared 
error;  i.e.,  (approximately)  one  standard  deviation  of  the  (approxi¬ 
mately)  normal  posterior  distribution  of  the  estimate  for  fixed 
frequency.  Figure  16  shows  the  corresponding  graphs  for  the 
spectral  density;  each  ordinate  in  Figure  15  is  just  the  natural 
logarithm  of  the  corresponding  ordinate  in  Figure  16. 
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APPENDIX 


i 


i 


Proof  of  Theorem  4.1. 
Set 


Znj  ■■■  ‘Vj  -  W1  ! 


then  E(Z^)  -  0,  VarU^)  -  1,  for  j  ■  0,1,  ...  ,n;  and 


Zqo  ,  ...  ,  Zm  are  Independent.  From  (2.7)  we  have 


U  »  ^  ir  .r _ (x  )$  (x  )  [Z  /  /X  ir  .)  +  R  (x  ,)-R  (x 

nr  L  Q  nj  no  nj  Tnr'  nj  nj  n  nj  n  nj  no  i 


and  from  (2.1)  and  (2.4)  we  have 


U  -  (1/  /T )  l  /ir  4r  (x  . )  $  (x  )Z.+  0  ,r-0 . 

nr  n  nj  no  nj  nr  nj  nj  nr* 


Then  from  (2.10),  (2.11),  and  (4.1)  , 


Vnr  "  ^_Q  /iV*nr(xnj)  rno(xnj)Znj  *  r"0,:L . n  * 


Let  f  .  be  the  characteristic  function  of  Z  .  ; 
nj  nj 


fnj(t)  E(exp(itZnj)]  ,  j  -  0,  1 . n. 


Let  k  be  an  Increasing  sequence  of  integers  satisfying  the 
n 


hypotheses  of  Theorem  4.1,  and  let  {a  . ,  j  -  0,1, . . .,  k  , 

nj  n 


kn 


n  -  0,1,  ...  }  be  an  array  of  real  numbers  such  that  7  a2  -1 

j-0  nj 

Since  Z  ,  ...  ,  Z  are  Independent,  it  follows  from 
no  nn  Jq 

..  n 

(A2)  that  the  characteristic  function  of  )  a  V  , 

r-0  nrar 


f  (t)  E (exp (it  l  a  V  )J  , 
n  “  n  nr  nr 


is  given  by 


f(t)  -  n  *  .«*“ r  (*  >  i  a  ♦  (x )jo  . 

n  jaQ  nj  nj  no  nj  zmQ  nr  nr  nJ 

For  fixed  t  ,  set 


'nj  =■  W  lImQ  mnr*nr(lnj)!t  ! 


f  (t>  -  n  £  4<t  j  . 


.0  “JnJ' 


Since  E^^)  •  0  and  VarCZ^j)  ■  1,  j  -  0,1,...  ,n, 


(  .(t)  -  1  -  tV2  +  (a „./6>|t|»  E|Z .  .  , 


where  lanj  I  <“  1*  and 


£nj  (V  -  1  +  9nj  • 


where 


9«J  -tnJ/2+<“nd/6)ltajl3BlZnjl3  ' 

Since  lanrH“  1  »  r  •  0,1,  ...  ,  kn  ,  by  Hypothesis  (4.3),  we  have 

®»x{  |  tnjl  ••  J  "  °»l9  •••  ,  n}  — ♦  0  as  n  — ►  »  , 

for  fixed  t  e  H .  And  according  to  (4.2),  E | znj I  3  M  for  all 
j  and  n ,  so  that 


{ | d  .|  :  j  ■  0,1,  ...  ,  n}  — »  0  as  n  — *  •  . 


v 


From  (2.4)  and  (A5)  , 

I  t2  *  t2  y  it  .r2  (x  .)  y  a  4>  (x  .)  7  a  <j>  (x  .) 
j.0  j-0  ^  n°  Qj  r-0  nr  nr  nj  s-0  ns  ns  nj 


so  that 


I  **  -  .*  . 


I  |e  .|  <-  t2/2  +  (M/6)  l  It  I* 

i-0  nj  <-0 


<  -  t2  /  2  +  (Mt2/6)  max{  (t^  [  :  j  -  0,1,  . . .  ,  n}  , 


so  that  by  (A9) ,  for  fixed  t  , 
n 

\  1 9  . |  Is  bounded. 

J-0  ^ 


Again,  from  (A8)  and  (All) , 


I*  V'2H<-  (Mt2  /  6)  max{|tnj|  :  j  -  0,1,  ...  ,n|  , 


so  that  by  (A9) 


M 

l  9  ~>  -t2/2  as  n  — >  •  . 

j-0  nJ 


It  follows  from  (A6>,  (A7),  (A10),  <A12) ,  and  (A13)  that 


f  (t) 
a 


n 


j-0 


f 

nj  Qj 


n 


j-0 


(1+9  ,)  — >  exp(-t2/2) 

aj 


as  n  -  (Chung,  1968,  page  184),  for  each  real  t  .  So 

>  .  V  convarga.  in  law  to  the  standard  normal  distribution, 

l  nr  nr 

and  the  array  V  is  jointly  asymptotically  normal.  D 


T 


r 


\  *  •  v 
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Figure  9.  256  points,  first  prior 
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Figure  10.  256  points,  fourth  prior 
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Figure  11.  256  points,  first  and  fourth  priors 
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Figure  12.  1,024  points,  first  prior 
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Figure  13.  1,024  points,  fourth  prior 
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Figure  14.  1,024  points,  first  and  fourth,  priors 
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Figure  16.  Spectral  density,  aabient  noise 


